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Previous work around school quality focused on identifying schools that were “beating the odds.” A typical 
approach has been to identify outlier schools by analyzing student outcomes, study these schools to determine 
what they appear to be doing well, and hopefully share these practices with other schools. In our early 
thinking about improving school quality, we began reframing quality around adult behaviors and inputs and the 
context they build in a school — factors we can leverage. This redefining of quality around adults resonated 
with us because of problems we perceive with approaches that place the most weight on student outcomes, 
which are biased by family poverty and factors outside of schools’ control. Next, we asked “If we have metrics on 
adults, can we predict the student outcomes used in previous accountability metrics?” Eventually this 


reframing could serve as a new nomenclature for quality and a model for guiding improvement. 


A review of the Effective Schools literature showed considerable overlap in the factors and conditions associated 
with schools “... in which students were mastering the curriculum at a higher rate and to a higher level than 
would be predicted based on students’ family background, gender, and racial and ethnic identification.” 
(National Center for Effective Schools Research and Development Foundation, 2003). A significant distinction 
between our work and the Effective Schools literature is that we separated student-related factors or outcomes 
from the predictors. Hence, we remove student outcomes from the formula so that the quality determination is 
not contingent on the characteristics of students in the school. After several theoretical iterations of the key 
adult factors (we call them drivers) affecting school quality, we finally reshaped our model to align with City 
Schools’s School Effectiveness Review Framework by adopting similar terms. Ultimately, this will facilitate 


alignment between BCF’s work and that of City Schools. The six drivers and their theoretical descriptions are: 


Safety: Students, teachers, and staff feel free of physical or psychological harm. 


Welcoming: Mutually welcoming and beneficial relationships are established between the school and 


parents/guardians. 


Instruction: Teacher creates a classroom environment that promotes learning, mutual respect, and 
high expectations. Teachers plan and deliver highly effective instruction that is relevant, accurate and 


promotes higher-level thinking. 


Teachers: Teachers have the professional capacity to deliver highly effective instruction and meet the 


learning needs of all students. 


Leaders: School leaders have the professional capacity to establish and communicate goals and 
strategic plans, serve as instructional leaders by supporting teacher growth, be responsive to and 
inclusive of staff and community, and provide the necessary structures to support the other primary 


drivers. 


External Engagement: Highly effective, sustained partnerships exist with relevant external 


community organizations and services. 
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We used this framework to create our driver diagram (see Figure 1). The purpose of the driver diagram is to 
establish the factors, that if appropriately leveraged, will achieve the desired aim(s). To be achievable, the aim 
should include a measurable outcome and specify a timeframe. All driver diagrams are incomplete and 
probably wrong, but are an important starting place to organize theoretical frameworks into a problem-solution 


diagram. 


As a team, we believe that by leveraging the six drivers we can achieve our aim to increase, by 50%, the number 
of quality seats by 2021-22. This project has allowed us to think about how to operationalize this model and 


develop prediction models examining student outcomes of attendance, academics, and behaviors. 


Figure 1. Driver diagram to increase number of quality seats provided by school 
or district. 


Aim Drivers 


Probably Wrong and Definitely Incomplete 


We implemented the driver diagram by identifying metrics for each of the drivers. We understood that we were 
limited by what data existed and also by what data were in BERC’s archive. In Table 1, we operationally 


describe the drivers and list the metrics used to measure each, across two years. 
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Table 1 


Driver Descriptions with Corresponding Year 1 and Year 2 Metrics 


Leaders 

Measures of the 
experience, knowledge, and 
skills leaders bring to the 
school. 


Year 1 Measures (2014-15) 


Administration factor from City 
Schools’ School Survey 


Year 2 Measures (2015-16) 


Administration factor from City 
Schools’ School Survey 


Principals’ years in position 


Teachers 

A measure of the 
experience, knowledge and 
skills that teachers bring to 
the classroom. 


% of teachers Certified 


% of Teachers missing fewer than 
10 days of school 


% of teachers Certified 


% of Teachers missing fewer than 
10 days of school 


Teachers’ average years- 
experience 


Safety 

Measures that indicate 
students feel safe and free 
from distractions to 
learning. 


Safety factor from City Schools’ 
School Survey 


Safety factor from City Schools’ 
School Survey 


Welcoming 

Measure of whether the 
school staff effectively 
communicates with and 
engages student families. 


Parent factor from City Schools’ 
School Survey 


Parent factor from City Schools’ 
School Survey 


Instruction 
Measures of how teachers 


structure their classroom 
environment, how they use 
instruction time, and 
promote academic rigor. 


External Partnerships 
Measures of whether school 
staff effectively 
communicate with 
community partners and 
harness community-based 
resources. 


Core Instruction factor from 
Student Survey on Teacher 
Practice 


Classroom Environment factor 
from Student Survey on Teacher 
Practice 


Teacher Effectiveness Scores: % of 
teachers at each school rated as 
effective or highly effective 


No data available 


No data available 
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Preliminary Findings 


Regression models control for total student enrollment; % LEP students; % FARMS-eligible students; % special 


education students; student-teacher ratio; % of students on benchmark in Amplify fall (for Elementary/Middle 


school models with spring Amplify outcomes); and entrance criteria (for secondary models). Drivers were 


included independently of each other in separate models, as several are highly correlated with each other. 


We began our analysis looking at elementary-only schools (schools serving pre-K/K to grade 5) and then 


elementary and middle schools. Per Tables 2 below, each of the five drivers had statistically significant 


relationships to one or more of the student outcomes, after controlling for characteristics of each school’s student 


body. The only student outcome tested that was not significantly related to at least one of the five drivers was 


student promotion to the next grade in the K-8 schools. Secondary school models provided far fewer statistically 


significant results due largely to the smaller sample size. There were 137 elementary/middle schools, but only 


38 secondary schools in the data set. Thus, in many cases, the regression models at the secondary level revealed 


very strong relationships between the drivers and student outcomes (with substantial effect sizes ranging from 


0.20 to 0.40), but statistical significance was not achieved due to the small sample size given the number of 


covariates included in the model. 


Table 2 


Statistically Significant Relationships Between Drivers and Outcomes for K-8 Schools 


Outcome 


Safety Welcoming | Instruction | Teachers Leaders 


Absent 


% of Students Chronically 


97% of School Da 


% of Students Attending over 


ys 


Failure 


% of Students with a Course 


% of Students Promoted 


PARCC ELA 


% of Students Meeting or 
Exceeding Expectations on 


PARCC Math 


% of Students Meeting or 
Exceeding Expectations on 


% of Students on 
— Spring Amplify 


Benchmark 


Key 


P<.05 in Year 1: 2014-15 


| | P<.05 in Year 2: 2015-16 a P<.05 in both Year 1 & 2 


4 |Thinking about Quality and Schools 


Quality and Schools 


Next Steps 


We hope to continue our analysis by further developing these models. This fall we will shift to and complete a 


more rigorous methodology to test and answer our questions. 


As a second methodological issue, the analysis of secondary schools did not yield the same findings. We 
hypothesize that this is because the model may require different drivers, 1.e., that different data would fit the 
model better. For example, challenges in secondary school might relate to developmental transitions, such as 


the growing influence of peers on youth and their own transition into adulthood. 


Related to the last point, a third consideration is that we are hampered by the limitation of available data. If 
the factors we believe are influencing quality are not measured, or data are not accessible, we are limited in how 
we might test our model and guide improvement efforts. For instance, more detailed human capital data might 
shed light on the instruction teachers bring to the classroom. High school models may better fit data like work- 


based learning opportunities and mentorship. 


Ultimately, given that the driver diagram has validity, it could shape or be formally integrated into existing 
dashboard metrics and used to guide improvement efforts around adult factors in school. Informally, the 


diagram could be used by school-level staff to develop their own aim, metrics, and change ideas. 
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